Uncertainty Modeling for Gaze Estimation

Abstract

Gaze estimation is an important fundamental task in computer vision and medical research. Existing works have explored various effective paradigms and modules for precisely predicting eye gazes. However, the uncertainty for gaze estimation, e.g., input uncertainty and annotation uncertainty, have been neglected in previous research. Existing models use a deterministic function to estimate the gaze, which cannot reflect the actual situation in gaze estimation.

To address this issue, we propose a probabilistic framework for gaze estimation by modeling the input uncertainty and annotation uncertainty.

Methodology

1. Input Uncertainty Modeling: We first utilize probabilistic embeddings to model the input uncertainty, representing the input image as a Gaussian distribution in the embedding space. Based on the input uncertainty modeling, we give an instance-wise uncertainty estimation to measure the confidence of prediction results, which is critical in practical applications.

2. Annotation Uncertainty Modeling: We propose a new label distribution learning method, probabilistic annotations, to model the annotation uncertainty, representing the raw hard labels as Gaussian distributions.

3. Embedding Distribution Smoothing (EDS): We develop an EDS module and a hard example mining method to improve the consistency between embedding distribution and label distribution.

Experimental Results

We conduct extensive experiments, demonstrating that the proposed approach achieves significant improvements over baseline and state-of-the-art methods on:

• GazeCapture dataset
• MPIIFaceGaze dataset
• Our collected dataset using mobile devices

The probabilistic framework successfully addresses the uncertainty challenges in gaze estimation, providing more reliable and accurate predictions with confidence measures.